Papers with internal mechanism
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Emerging jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment. |
| Approach: | They propose to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other, semantically-dissimilar classes. |
| Outcome: | The proposed jailbreak vectors show that they reduce harmfulness in most models, and that they are similar in geometric and effect similarity. |
The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on the inputoutput behavior of LMs, leaving the internal mechanisms behind ICL largely unexplored. |
| Approach: | They investigate how different prompting methods modify internal representations in pre-trained language models. |
| Outcome: | The proposed model can be programmed with natural language to perform a wide array of tasks without expensive fine-tuning. |
Detoxifying Large Language Models via Knowledge Editing (2024.acl-long)
Copied to clipboard
Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, Huajun Chen
| Challenge: | Existing methods to detoxify Large Language Models (LLMs) are limiting, but knowledge editing can be effective. |
| Approach: | They propose a baseline method to detoxify Large Language Models (LLMs) they propose supervised fine-tuning and reinforcement learning from human feedback (RLHF) |
| Outcome: | The proposed method reduces toxicity of large language models with one instance of tuning . it reduces the toxicity, while minimizing the toxins, the authors show . |
Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models generate inconsistent and sometimes contradictory outputs when presented with a prompt that has equivalent semantics but is expressed differently from the original prompt. |
| Approach: | They propose to refine a Large Language Model (LLM) with prompt-output pairs with equivalent semantics to achieve semantic consistency. |
| Outcome: | The proposed method improves the semantic consistency and task performance of LLMs. |
ReLoop: “Seeing Twice and Thinking Backwards” via Closed-loop Training to Mitigate Hallucinations in Multimodal understanding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for hallucination mitigation rely on external verification or post-hoc correction, lacking internal mechanism to validate outputs directly during training. |
| Approach: | They propose a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. |
| Outcome: | The proposed framework encourages multimodal consistency for cross-modal understanding in MLLMs. |
Investigating Value-Reasoning Reliability in Small Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | sLLMs have been widely deployed in practical applications, but little attention has been paid to their value-reasoning abilities, particularly in terms of reasoning reliability. |
| Approach: | They propose a systematic evaluation framework for assessing the Value-Reasoning Reliability of small Large Language models (sLLMs) . framework includes three core tasks: Repetition Consistency task, Interaction Stability task, and Open-ended Expression Consistencies task. |
| Outcome: | The proposed framework incorporates self-reported confidence scores to evaluate the model’s value reasoning reliability from two perspectives: the model's self awareness of its values, and its value-based decision-making. |
One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them (2026.findings-acl)
Copied to clipboard
| Challenge: | Knowledge editing methods such as ROME and MEMIT update factual associations by modifying MLP weights. |
| Approach: | They propose to use a mask to reverse edits by eliminating overattention in later layers . they also show that injecting the mask during editing drops editing success from 98% to 38% . |
| Outcome: | The proposed method reverses edits by eliminating overattention in later layers and drops editing success from 98% to 38%. |
Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them (2025.findings-emnlp)
Copied to clipboard
Guanyu Chen, Peiyang Wang, Yizhou Jiang, Yuqian Liu, Chujie Zhao, Ying Fang, Tianren Zhang, Feng Chen
| Challenge: | Existing large language models can perform abstract reasoning tasks but are they actually engaging in rule-based reasoning beyond mere memorization? |
| Approach: | They propose a method to examine whether large language models perform abstract reasoning . they fine-tune the model to learn those contradictory rules and assess its generalization ability . |
| Outcome: | The proposed approach examines whether large language models perform abstract reasoning by altering their original understanding of fundamental rules. |